Method and apparatus for tracking eyes of user and method of generating inverse-transform image

ABSTRACT

There is provided a method and apparatus for tracking eyes of a user. The method and apparatus may acquire an image of the user, acquire an illuminance of a viewpoint from which the image is captured, and output coordinates of the eyes tracked from the image by operating at least one of a high illuminance eye tracker that operates at a high illuminance or a low illuminance eye tracker that operates at a low illuminance based on the acquired illuminance.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.15/810,730, filed on Nov. 13, 2017, which claims priority from KoreanPatent Application No. 10-2016-0155600, filed on Nov. 22, 2016, in theKorean Intellectual Property Office, the disclosure of which isincorporated herein by reference in its entirety.

BACKGROUND 1. Field

Methods and apparatuses consistent with exemplary embodiments in thisdisclosure relate to a method and an apparatus for tracking eyes of auser, and more particularly, to a method of generating aninverse-transform image and an apparatus for generating aninverse-transform image.

2. Description of the Related Art

Related art camera-based eye tracking technology may be utilized in manyfields, such as an ultra multi-view glasses-free three-dimensional (3D)display based on viewpoint tracking. The ultra multi-view glasses-freethree-dimensional (3D) display may be used in a low lighted area (darklocation) as well as a well-lit area (bright location), while watching atelevision (TV) or using a mobile device. However, while the related artcamera-based eye tracker may properly operate in a high illuminanceenvironment (for example, 400 lux), the related art camera-based eyetracker may not properly operate in a low illuminance environment due toa low quality of a camera image.

Also, in a next generation 3D head-up display (HUD) technology forvehicles, night driving may need to be taken into consideration.Accordingly, there is a desire for an eye tracker that properly operatesat a low illuminance.

SUMMARY

Exemplary embodiments may address at least the above problems and/ordisadvantages and other disadvantages not described above. Also, theexemplary embodiments are not required to overcome the disadvantagesdescribed above, and an example embodiment may not overcome any of theproblems described above.

According to an aspect of an exemplary embodiment, there is provided amethod of tracking eyes of an object, the method may comprise: acquiringan image of the object, acquiring an illuminance of a viewpoint fromwhich the image is captured and outputting coordinates of the eyestracked from the image by operating at least one of a first illuminanceeye tracker that operates at a first illuminance range or a secondilluminance eye tracker that operates at a second illuminance rangebased on the acquired illuminance.

The first illuminance eye tracker may be trained by machine learningbased on a first illuminance database (DB) comprising first illuminanceimages captured at the first illuminance range, and the secondilluminance eye tracker may be trained by machine learning based on asecond illuminance DB comprising inverse-transform images obtained byinversely transforming the first illuminance images into secondilluminance images.

The inverse-transform images may comprise feature points trained by themachine learning based on the first illuminance DB.

The second illuminance DB may comprise the inverse-transform imagesgenerated by applying, to the first illuminance images, an imagehistogram characteristic of a second illuminance reference image modeledin advance based on second illuminance images captured at a secondilluminance range.

The second illuminance DB may comprise the inverse-transform imagesgenerated by applying, to the first illuminance images, a noisecomponent of the second illuminance reference image in addition to theimage histogram characteristic.

Each of the first illuminance eye tracker and the second illuminance eyetracker may comprise a plurality of classifiers configured to recognizefeature points corresponding to eyes of the object from the image.

The outputting of the coordinates may comprise comparing the acquiredilluminance to a preset reference value, outputting the coordinates byoperating the second illuminance eye tracker when the acquiredilluminance is less than or equal to the reference value and outputtingthe coordinates by operating the first illuminance eye tracker when theacquired illuminance is greater than the reference value.

The reference value may be set to 10 lux.

The acquiring of the image may comprise capturing the image using asingle image sensor.

According to an aspect of another embodiment, there is provided a methodof generating an inverse-transform image, the method may comprise:modeling a first illuminance reference image based on first illuminanceimages captured at a first illuminance, acquiring an image histogramcharacteristic of the first illuminance reference image, generating aintermediate image by adjusting a brightness level in an image histogramof a second illuminance image captured at a second illuminance andinversely transforming the intermediate image into a first illuminanceimage by applying the image histogram characteristic to the intermediateimage.

The method may further comprise acquiring a noise component of the firstilluminance reference image.

The inversely transforming of the intermediate image may compriseinversely transforming the intermediate image into the first illuminanceimage by applying the noise component and the image histogramcharacteristic to the intermediate image.

The noise component may comprise at least one of Gaussian noise of theilluminance reference image or Poisson noise of the first illuminancereference image.

According to an aspect of another embodiment, there is provided anon-transitory computer-readable storage medium storing a program forcausing a processor to perform the method of generating aninverse-transform image, the method may comprise: modeling a firstilluminance reference image based on first illuminance images capturedat a first illuminance, acquiring an image histogram characteristic ofthe first illuminance reference image, generating a intermediate imageby adjusting a brightness level in an image histogram of a secondilluminance image captured at a second illuminance and inverselytransforming the intermediate image into a first illuminance image byapplying the image histogram characteristic to the intermediate image.

According to an aspect of another embodiment, there is provided anapparatus for tracking eye of an object, the apparatus may comprise animage sensor configured to capture an image of the object, anilluminance sensor configured to measure an illuminance of a viewpointfrom which the image is captured, a first illuminance eye trackerconfigured to operate at a first illuminance range, a second illuminanceeye tracker configured to operate at a second illuminance range and aprocessor configured to control coordinates of the eyes tracked from theimage to be output by operating at least one of the first illuminanceeye tracker or the second illuminance eye tracker based on the measuredilluminance.

The first illuminance eye tracker may be trained by machine learningbased on a first illuminance database (DB) comprising first illuminanceimages captured at the first illuminance range, and the secondilluminance eye tracker may be trained by machine learning based on asecond illuminance DB comprising inverse-transform images obtained byinversely transforming the first illuminance images into secondilluminance images.

The second illuminance DB may comprise the inverse-transform imagesgenerated by applying, to the first illuminance images, an imagehistogram characteristic of a second illuminance reference image modeledin advance based on second illuminance images captured at a secondilluminance range.

The illuminance DB may comprise the inverse-transform images generatedby applying, to the first illuminance images, a noise component of thesecond illuminance reference image in addition to the image histogramcharacteristic.

According to an aspect of another embodiment, there is provided anilluminance eye tracker comprising: a illuminance database (DB)comprising inverse-transform images obtained by inversely transformingfirst illuminance images into second illuminance images; and an imageprocessor configured to process a second illuminance image of an objectby a parameter trained by machine learning based on the illuminance DB,and to output coordinates of eyes of the object.

The illuminance DB may comprise the inverse-transform images generatedby applying, to the first illuminance images, an image histogramcharacteristic of a second illuminance reference image modeled inadvance based on second illuminance images captured at a secondilluminance range.

The illuminance DB may comprise the inverse-transform images generatedby applying, to the first illuminance images, a noise component of thesecond illuminance reference image in addition to the image histogramcharacteristic.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects of exemplary embodiments will becomeapparent and more readily appreciated from the following detaileddescription of certain example embodiments, with reference to theaccompanying drawings of which:

FIG. 1 is a flowchart illustrating a method of tracking eyes of a useraccording to an exemplary embodiment;

FIG. 2 is a flowchart illustrating an example of an operation ofoutputting coordinates of eyes of a user in the method of FIG. 1;

FIG. 3 is a diagram illustrating a concept of a method of generating aninverse-transform image according to an exemplary embodiment;

FIG. 4 is a diagram illustrating a method of generating aninverse-transform image according to an exemplary embodiment;

FIG. 5 is a flowchart illustrating a method of generating a lowilluminance database (DB) including an inverse-transform image accordingto an exemplary embodiment;

FIG. 6 is a block diagram illustrating an apparatus for tracking eyes ofa user according to an exemplary embodiment; and

FIG. 7 is a block diagram illustrating a low illuminance eye trackeraccording to an exemplary embodiment.

DETAILED DESCRIPTION

Reference will now be made in detail to exemplary embodiments, examplesof which are illustrated in the accompanying drawings, wherein likereference numerals refer to the like elements throughout. Exemplaryembodiments are described below in order to explain the presentdisclosure by referring to the figures.

The following structural or functional descriptions are exemplary tomerely describe the exemplary embodiments, and the scope of theexemplary embodiments is not limited to the descriptions provided in thepresent specification. Various changes and modifications can be madethereto by those of ordinary skill in the art.

Although terms of “first” or “second” are used to explain variouscomponents, the components are not limited to the terms. These termsshould be used only to distinguish one component from another component.For example, a “first” component may be referred to as a “second”component, or similarly, and the “second” component may be referred toas the “first” component within the scope of the right according to theconcept of the present disclosure.

It will be understood that when a component is referred to as being“connected to” another component, the component can be directlyconnected or coupled to the other component or intervening componentsmay be present.

As used herein, the singular forms are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It shouldbe further understood that the terms “comprises” and/or “comprising,”when used in this specification, specify the presence of statedfeatures, integers, steps, operations, elements, components or acombination thereof, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

Unless otherwise defined herein, all terms used herein includingtechnical or scientific terms have the same meanings as those generallyunderstood by one of ordinary skill in the art. Terms defined indictionaries generally used should be construed to have meaningsmatching with contextual meanings in the related art and are not to beconstrued as an ideal or excessively formal meaning unless otherwisedefined herein.

In the following description, exemplary embodiments may describe methodsand apparatuses used to track eyes of a user and output coordinates ofthe eyes when a glasses-free three-dimensional (3D) monitor, aglasses-free 3D tablet and/or smartphone, or a 3D head-up display (HUD)for vehicles is used in a dark environment with a relatively lowilluminance. Also, exemplary embodiments may be implemented as, forexample, a software algorithm in a chip or a hardware processor in amonitor, or an application in a tablet and/or smartphone, and may beimplemented as a hardware eye tracking apparatus. For example, exampleembodiments may be applicable to an autonomous vehicle, an intelligentvehicle, a smartphone or a mobile device. Hereinafter, exemplaryembodiments will be described in detail below with reference to theaccompanying drawings, and like reference numerals refer to the likeelements throughout.

FIG. 1 is a flowchart illustrating a method of tracking eyes of a useraccording to an exemplary embodiment. Referring to FIG. 1, in operation110, an apparatus for tracking eyes of a user (hereinafter, referred toas a “tracking apparatus”) according to an exemplary embodiment acquiresan image of the user. The image may include, for example, a face imageof the user or a body image including a face of the user.

According to an exemplary embodiment, the tracking apparatus may acquirean image of a user that is directly captured by an image sensor or animaging device included in the tracking apparatus. The trackingapparatus may capture an image of a user using a single imaging deviceor a single age sensor. Also, the tracking apparatus may receive animage of a user captured outside the tracking apparatus. The imagesensor may include, for example, a complementarymetal-oxide-semiconductor (CMOS) image sensor or a vision sensor.

In operation 130, the tracking apparatus acquires an illuminance of aviewpoint from which the image is captured. For example, the trackingapparatus may acquire an illuminance sensed by an imaging device thatcaptures the image and/or an illuminance sensed at a location of theuser. Also, the tracking apparatus may sense an illuminance by anilluminance sensor included in the tracking apparatus, or may receive asensing result from a separate illuminance sensor outside the trackingapparatus.

In operation 150, the tracking apparatus outputs coordinates of the eyestracked from the image by operating at least one of a high illuminanceeye tracker or a low illuminance eye tracker based on the acquiredilluminance. The high illuminance eye tracker may operate at a highilluminance, and the low illuminance eye tracker may operate at a lowilluminance. Each of the high illuminance eye tracker and the lowilluminance eye tracker may include a plurality of classifiersconfigured to recognize feature points corresponding to the eyes fromthe image.

A classifier may recognize landmarks, that is, feature pointscorresponding to eyes and a nose of a user from a face area of an imageof the user using, for example, an active shape model (ASM) scheme, anactive appearance model (AAM) scheme or a supervised descent method(SDM).

According to another embodiment, the tracking apparatus may include aplurality of classifiers configured to recognize other feature points inthe image, i.e., landmarks other than eyes and a nose from the image fortracking.

For example, the high illuminance eye tracker may track eyes of a userby classifiers pre-trained to recognize feature points corresponding tothe eyes and a nose of the user based on the eyes from a face of theuser, and may output coordinates of the eyes. According to an exemplaryembodiment, the feature points acquired by the high illuminance eyetracker may be used and trained for a low illuminance image obtained byinversely transforming a high illuminance image. The feature pointsacquired by the high illuminance eye tracker may be used and trainedwithout a change for a low illuminance image obtained by inverselytransforming a high illuminance image. In the present disclosure, a lowilluminance image obtained by inversely transforming a high illuminanceimage may be referred to as an “inverse-transform image.”Inverse-transform images included in a low illuminance database (DB) mayinclude the same feature points as those trained by machine learningbased on a high illuminance DB.

For example, machine learning may be performed by, for example, anadaptive boosting (AdaBoost) or a support vector machine (SVM) scheme,however, there is no limitation thereto. Accordingly, various learningmethods may be applied.

The low illuminance DB may include inverse-transform images generated byapplying, to high illuminance images, an image histogram characteristicof a low illuminance reference image modeled in advance based on lowilluminance images captured at a low illuminance. Also, the lowilluminance DB may include inverse-transform images generated byapplying, to high illuminance images, a noise component of the lowilluminance reference image in addition to the image histogramcharacteristic.

Hereinafter, an example of operation 150 will be further described withreference to FIG. 2.

FIG. 2 is a flowchart illustrating an example of operation 150 ofFIG. 1. Referring to FIG. 2, in operation 210, the tracking apparatuscompares the acquired illuminance to a preset reference value. Thereference value may be set to, for example, 10 lux.

When the acquired illuminance is less than or equal to the referencevalue, the tracking apparatus may output the coordinates by operatingthe low illuminance eye tracker in operation 220. For example, when theacquired illuminance is less than or equal to 10 lux that is thereference value, the tracking apparatus may determine the acquiredilluminance as a low illuminance, and may operate the low illuminanceeye tracker. The low illuminance eye tracker may be trained by machinelearning based on a low illuminance DB including inverse-transformimages. The low illuminance eye tracker may process a low illuminanceimage of a user by a parameter trained by machine learning based on thelow illuminance DB, and may output coordinates of eyes of the user.

When the acquired illuminance is greater than the reference value, thetracking apparatus may output the coordinates by operating the highilluminance eye tracker in operation 230. For example, when the acquiredilluminance is greater than 10 lux that is the reference value, thetracking apparatus may determine the acquired illuminance as a highilluminance and may operate the high illuminance eye tracker. The highilluminance eye tracker may process a high illuminance image of a userby a parameter trained by machine learning based on the high illuminanceDB, and may output coordinates of eyes of the user.

FIG. 3 is a flow chart illustrating a method of generating aninverse-transform image according to an exemplary embodiment. FIG. 3illustrates an image enhancement process performed by operations 301,303, 305 and 307, and a process of generating an inverse-transform imagethrough operations 310 and 330.

An inverse-transform image may be generated by inversely performing theimage enhancement process, and accordingly the image enhancement processwill be described prior to the process of generating aninverse-transform image.

The image enhancement process may be an image processing scheme toenhance a dark and blurred image or an input unclear image caused bynoise to be more suitable image for a special application purpose, tofacilitate recognition. The image enhancement process may broadlyinclude operations 301 through 307.

For example, when a low illuminance image is input in operation 301, ahistogram equalization to allow the low illuminance image to be clearlyviewed may be performed in operation 303. In the histogram equalization,pixel values of an image may be uniformly distributed to fill an entirearea of the image by a contrast adjustment so that the image may beclarified. For example, in the histogram equalization, a uniformfunction or a Gaussian function may be used.

When the histogram equalization is performed, noise may be removed fromthe low illuminance image in operation 305. In operation 307, anenhanced image, that is, a high illuminance image may be output. Thenoise may be removed using various noise filters or various algorithmsfor removing noise, for example, a radiometric correction.

An inverse transformation may be performed by inversely performing aprocess of generating a high illuminance image from the above-describedlow illuminance image, to generate a low illuminance image from a highilluminance image.

For example, when the high illuminance image is input, a generationapparatus may add noise to the high illuminance image in operation 310.In this example, the noise may be noise of an average low illuminanceimage. The noise of the average low illuminance image may be, forexample, Gaussian noise or Poisson noise of the average low illuminanceimage. The Gaussian noise may correspond to general noise with a normaldistribution. The Poisson noise may correspond to Poisson distributionnoise.

In operation 330, the generation apparatus may perform a histogramspecification of the high illuminance image with the noise and mayinversely transform the high illuminance image into a low illuminanceimage. The histogram specification may refer to transformation formatching a histogram of a corresponding image and a predeterminedhistogram. The generation apparatus may inversely transform the highilluminance image by applying an image histogram characteristic of animage captured at a low illuminance to the high illuminance image. Forexample, the generation apparatus may transform a histogram of the highilluminance image with the noise so that the histogram of the highilluminance image with the noise may match a histogram of a lowilluminance image, and may generate an inverse-transform image from thehigh illuminance image. An example of a method of configuring a lowilluminance DB by generating an inverse-transform image in thegeneration apparatus will be further described with reference to FIGS. 4and 5.

A low illuminance image may be generated by inversely transforming ahigh illuminance image, and thus it is possible to configure a lowilluminance DB for training of a low illuminance eye tracker based onhigh illuminance images stored in a high illuminance DB even thoughseparate low illuminance images are absent.

FIG. 4 is a diagram illustrating a method of generating aninverse-transform image according to an exemplary embodiment. Referringto FIG. 4, a plurality images, for example, a high illuminance image410, a low illuminance reference image 420, a dark image 430 and aninverse-transform image 440, represent a face of a user. Also, graphs415, 425, 435 and 445 show image histograms corresponding to the highilluminance image 410 through the inverse-transform image 440,respectively.

The high illuminance image 410 may be a high illuminance image capturedat a high illuminance, for example, 450 lux.

The low illuminance reference image 420 may be an image modeled based onlow illuminance images captured at a low illuminance, for example, 10lux.

The dark image 430 may be an image generated by adjusting a brightnesslevel to a low illuminance, for example, 10 lux, in an image histogramof the high illuminance image 410.

The inverse-transform image 440 may be a low illuminance image obtainedby inversely transforming the dark image 430 by applying an imagehistogram characteristic of the low illuminance reference image 420 tothe dark image 430.

As described above, an inverse-transform image included in a lowilluminance DB according to an exemplary embodiment may be generated byinversely transforming a high illuminance image into a low illuminanceimage. Hereinafter, a method of inversely transforming a highilluminance image into a low illuminance image will be furtherdescribed.

A generation apparatus may model the low illuminance reference image 420based on a plurality of low illuminance images captured at a lowilluminance, for example, 10 lux, and may acquire an image histogramcharacteristic of the low illuminance reference image 420 as shown inthe graph 425. The graph 425 corresponds to the image histogramcharacteristic of the low illuminance reference image 420, and indicatesthat relatively large number of pixels are concentrated in an intensityless than or equal to “125.”

The generation apparatus may generate the dark image 430 from the highilluminance image 410. The generation apparatus may generate the darkimage 430 by adjusting the brightness in the image histogram of the highilluminance image 410. For example, the generation apparatus may adjustthe brightness to be an intensity less than or equal to “125” byshifting the histogram corresponding to the high illuminance image 410leftwards in the graph 415, and may generate the dark image 430corresponding to the graph 435.

The generation apparatus may generate the inverse-transform image 440 byapplying the image histogram characteristic of the low illuminancereference image 420 shown in the graph 425 to the dark image 430. Animage histogram of the inverse-transform image 440 may be obtained bycombining an image histogram characteristic of the dark image 430 andthe image histogram characteristic of the low illuminance referenceimage 420, as shown in the graph 445.

FIG. 5 is a flowchart illustrating a method of generating a lowilluminance DB including an inverse-transform image according to anexemplary embodiment. Referring to FIG. 5, in operation 510, anapparatus for generating a low illuminance DB (hereinafter, referred toas a “generation apparatus”) according to an exemplary embodimentperforms modeling of a low illuminance reference image based on lowilluminance images captured at a low illuminance. For example, thegeneration apparatus may perform modeling of a low illuminance referenceimage based on a plurality of low illuminance images captured at 10 lux.The low illuminance reference image may be modeled for each object. Forexample, the low illuminance reference image may be modeled fordifferent types or for different objects, for example, a human, a cat ora dog. The low illuminance reference image may be modeled as, forexample, the low illuminance reference image 420 of FIG. 4.

In operation 520, the generation apparatus acquires an image histogramcharacteristic of the low illuminance reference image. The imagehistogram characteristic of the low illuminance reference image may berepresented, for example, as shown in the graph 425 of FIG. 4.

In operation 530, the generation apparatus generates a dark image byadjusting a brightness level in an image histogram of a high illuminanceimage captured at a high illuminance. For example, the generationapparatus may adjust the brightness to be a low illuminance by shiftingthe image histogram of the high illuminance image, and may generate thedark image.

In operation 540, the generation apparatus inversely transforms the darkimage into a low illuminance image by applying the image histogramcharacteristic of the low illuminance reference image to the dark image,to generate an inverse-transform image.

For example, the generation apparatus may further acquire a noisecomponent of the low illuminance reference image. In this example, thegeneration apparatus may apply the noise component and the imagehistogram characteristic of the low illuminance reference image to thedark image generated in operation 530, to inversely transform the darkimage into a low illuminance image. The noise component may include, forexample, Gaussian noise and Poisson noise of the low illuminancereference image.

In operation 550, the generation apparatus stores the inverse-transformimage in the low illuminance DB.

FIG. 6 is a block diagram illustrating a tracking apparatus 600according to an exemplary embodiment. Referring to FIG. 6, the trackingapparatus 600 includes an image sensor 610, an illuminance sensor 620, aprocessor 630, a low illuminance eye tracker 640 and a high illuminanceeye tracker 650. The tracking apparatus 600 may further include a memory660.

The image sensor 610 may capture an image of a user.

The illuminance sensor 620 may measure an illuminance of a viewpointfrom which the image is captured.

The processor 630 may control coordinates of eyes of the user trackedfrom the image to be output by operating at least one of the lowilluminance eye tracker 640 or the high illuminance eye tracker 650based on the measured illuminance.

Also, the processor 630 may perform at least one of the methodsdescribed above with reference to FIGS. 1 through 5. According to anexemplary embodiment, the processor 630 may perform the functions of thegeneration apparatus illustrated with respect to FIGS. 3, 4 and 5. Theprocessor 630 may execute a program and may control the trackingapparatus 600. A program code executed by the processor 630 may bestored in the memory 660.

The low illuminance eye tracker 640 may operate at a low illuminance, totrack positions of the eyes from the image and to output coordinates ofthe eyes. The low illuminance eye tracker 640 may be trained by machinelearning based on a low illuminance DB including inverse-transformimages obtained by inversely transforming high illuminance images. Forexample, the low illuminance DB may include inverse-transform imagesgenerated by applying, to high illuminance images, an image histogramcharacteristic of a low illuminance reference image modeled in advancebased on low illuminance images captured at a low illuminance. Also, thelow illuminance DB may include inverse-transform images generated byapplying, to high illuminance images, a noise component of the lowilluminance reference image in addition to the image histogramcharacteristic of the low illuminance reference image.

The high illuminance eye tracker 650 may operate at a high illuminance,to track positions of the eyes from the image and to output coordinatesof the eyes. The high illuminance eye tracker 650 may be trained bymachine learning based on a high illuminance DB including highilluminance images captured at the high illuminance.

The memory 660 may store the image captured by the image sensor 610and/or the coordinates of the eyes output by the low illuminance eyetracker 640 and the high illuminance eye tracker 650. The memory 660 mayinclude the low illuminance DB and the high illuminance DB depending onexample embodiments.

Also, the memory 660 may store a variety of information generated in theabove-described processing process of the processor 630. Furthermore,the memory 660 may store a variety of data and programs. The memory 660may include, for example, a volatile memory or a nonvolatile memory. Thememory 660 may include a mass storage medium, for example, a hard disk,to store a variety of data.

FIG. 7 is a block diagram illustrating a low illuminance eye tracker 700according to an exemplary embodiment. Referring to FIG. 7, the lowilluminance eye tracker 700 includes a low illuminance DB 710 and animage processor 730.

The low illuminance DB 710 may include inverse-transform images obtainedby inversely transforming high illuminance images. For example, the lowilluminance DB 710 may include inverse-transform images generated byapplying, to high illuminance images, an image histogram characteristicof a low illuminance reference image modeled in advance based on lowilluminance images captured at a low illuminance. Also, the lowilluminance DB 710 may include inverse-transform images generated byapplying, to high illuminance images, a noise component of the lowilluminance reference image in addition to the image histogramcharacteristic of the low illuminance reference image. The lowilluminance DB 710 may be included in, for example, a memory (notshown).

The image processor 730 may process a low illuminance image of a user bya parameter trained in advance by machine learning based on the lowilluminance DB 710, and may output coordinates of eyes of the user. Forexample, the coordinates output by the image processor 730 may bedirectly used as an input of a glasses-free 3D display.

Also, the image processor 730 may perform at least one of the methodsdescribed above with reference to FIGS. 3, 4 and 5. The image processor730 may execute a program and may control the low illuminance eyetracker 700. According to an exemplary embodiment, the processor 730 mayperform the functions of the generation apparatus illustrated withrespect to FIGS. 3, 4 and 5. A program code executed by the lowilluminance eye tracker 700 may be stored in a memory.

The exemplary embodiments described herein may be implemented usinghardware components, software components, or a combination thereof. Aprocessing device may be implemented using one or more general-purposeor special purpose computers, such as, for example, a processor, acontroller and an arithmetic logic unit, a digital signal processor, amicrocomputer, a field programmable array, a programmable logic unit, amicroprocessor or any other device capable of responding to andexecuting instructions in a defined manner. The processing device mayrun an operating system (OS) and one or more software applications thatrun on the OS. The processing device also may access, store, manipulate,process, and create data in response to execution of the software. Forpurpose of simplicity, the description of a processing device is used assingular; however, one skilled in the art will appreciated that aprocessing device may include multiple processing elements and multipletypes of processing elements. For example, a processing device mayinclude multiple processors or a processor and a controller. Inaddition, different processing configurations are possible, such aparallel processors.

The software may include a computer program, a piece of code, aninstruction, or some combination thereof, to independently orcollectively instruct or configure the processing device to operate asdesired. Software and data may be embodied permanently or temporarily inany type of machine, component, physical or virtual equipment, computerstorage medium or device, or in a propagated signal wave capable ofproviding instructions or data to or being interpreted by the processingdevice. The software also may be distributed over network coupledcomputer systems so that the software is stored and executed in adistributed fashion. The software and data may be stored by one or morenon-transitory computer readable recording mediums.

The method according to the above-described exemplary embodiments may berecorded in non-transitory computer-readable media including programinstructions to implement various operations which may be performed by acomputer. The media may also include, alone or in combination with theprogram instructions, data files, data structures, and the like. Theprogram instructions recorded on the media may be those speciallydesigned and constructed for the purposes of the example embodiments, orthey may be of the well-known kind and available to those having skillin the computer software arts. Examples of non-transitorycomputer-readable media include magnetic media such as hard disks,floppy disks, and magnetic tape; optical media such as CD ROM discs andDVDs; magneto-optical media such as optical discs; and hardware devicesthat are specially configured to store and perform program instructions,such as read-only memory (ROM), random access memory (RAM), flashmemory, and the like. Examples of program instructions include bothmachine code, such as code produced by a compiler, and files containinghigher level code that may be executed by the computer using aninterpreter. The described hardware devices may be configured to act asone or more software modules in order to perform the operations of theabove-described example embodiments, or vice versa.

While this disclosure includes exemplary embodiments, it will beapparent to one of ordinary skill in the art that various changes inform and details may be made in these exemplary embodiments withoutdeparting from the spirit and scope of the claims and their equivalents.The exemplary embodiments described herein are to be considered in adescriptive sense only, and not for purposes of limitation. Descriptionsof features or aspects in each exemplary embodiment are to be consideredas being applicable to similar features or aspects in other examples.Suitable results may be achieved if the described techniques areperformed in a different order, and/or if components in a describedsystem, architecture, device, or circuit are combined in a differentmanner and/or replaced or supplemented by other components or theirequivalents. Therefore, the scope of the disclosure is defined not bythe detailed description, but by the claims and their equivalents, andall variations within the scope of the claims and their equivalents areto be construed as being included in the disclosure.

What is claimed is:
 1. A method of tracking eyes of an object, themethod comprising: acquiring an image of the object; determining anilluminance of an environment at which the image is captured; andoutputting coordinates of the eyes tracked from the image by operatingat least one of a first illuminance eye tracker that operates at a firstilluminance range or a second illuminance eye tracker that operates at asecond illuminance range based on the illuminance.
 2. The method ofclaim 1, wherein the first illuminance range has a higher finance valuethan a second illuminance range.
 3. The method of claim 1, wherein thefirst illuminance eye tracker is a high illuminance eye tracker and thesecond illuminance eye tracker is a low illuminance eye tracker, thehigh illuminance eye tracker is different from the low illuminance eyetracker and the high illuminance eye tracker operates at a higherilluminance value than the low illuminance eye tracker.
 4. The method ofclaim 1, wherein the coordinates of the eyes tracked from the image areoutput by operating the first illuminance eye tracker based on theilluminance being greater than a threshold value, and the coordinates ofthe eyes tracked from the image are output by operating the secondilluminance eye tracker based on the illuminance being lower than thethreshold value.
 5. The method of claim 4, wherein the threshold valueis set to 10 lux.
 6. The method of claim 1, wherein the firstilluminance eye tracker is trained by machine learning based on a firstilluminance database (DB) comprising first illuminance images of thefirst illuminance range.
 7. The method of claim 1, wherein the secondilluminance eye tracker is trained by machine learning based on a secondilluminance DB comprising second illuminance images of the secondilluminance range.
 8. The method of claim 1, wherein the acquiring ofthe image comprises capturing the image using a single image sensor. 9.A non-transitory computer-readable storage medium storing a program forcausing a processor to perform the method of claim
 1. 10. A method oftracking eyes of an object, the method comprising: capturing an image ofthe object using an image sensor; measuring an illuminance of anenvironment using an illuminance sensor; and outputting coordinates ofthe eyes tracked from the image by operating at least one of a firstilluminance eye tracker that operates at a first illuminance range or asecond illuminance eye tracker that operates at a second illuminancerange based on the illuminance.
 11. The method of claim 10, wherein thefirst illuminance range has a higher illuminance value than a secondilluminance range.
 12. The method of claim 10, wherein the firstilluminance eye tracker is a high illuminance eye tracker and the secondilluminance eye tracker is a low illuminance eye tracker, the highilluminance eye tracker is different from the low illuminance eyetracker and the high illuminance eye tracker operates at a higherilluminance value than the low illuminance eye tracker.
 13. The methodof claim 10, wherein the coordinates of the eyes tracked from the imageare output by operating the first illuminance eye tracker based on theilluminance being greater than a threshold value, and the coordinates ofthe eyes tracked from the image are output by operating the secondilluminance eye tracker based on the illuminance being lower than thethreshold value.
 14. The method of claim 13, wherein the threshold valueis set to 10 lux.
 15. The method of claim 10, wherein the firstilluminance eye tracker is trained by machine learning based on a firstilluminance database (DB) comprising first illuminance images of thefirst illuminance range.
 16. The method of claim 10, wherein the secondilluminance eye tracker is trained by machine learning based on a secondilluminance DB comprising second illuminance images of the secondilluminance range.
 17. The method of claim 10, wherein the acquiring ofthe image comprises capturing the image using a single image sensor. 18.A non-transitory computer-readable storage medium storing a program forcausing a processor to perform the method of claim
 10. 19. An apparatusfor tracking eyes of an object, the apparatus comprising: an imagesensor configured to capture an image of the object; and a processorconfigured to determine an illuminance of an environment at which theimage is captured, and to control coordinates of the eyes tracked fromthe image to be output by operating at least one of the firstilluminance eye tracker that operates at a first illuminance range orthe second illuminance eye tracker that operates at a second illuminancerange based on the illuminance.
 20. An apparatus for tracking eyes of anobject, the apparatus comprising: an image sensor configured to capturean image of the object; an illuminance sensor configured to measure anilluminance of an environment; and a processor configured to controlcoordinates of the eyes tracked from the image to be output by operatingat least one of the first illuminance eye tracker that operates at afirst illuminance range or the second illuminance eye tracker thatoperates at a second illuminance range based on the illuminance.